Action Selection in Customer Value Optimization: An Approach Based on Covariate-Dependent Markov Decision Processes

Typical methods in CRM marketing include action selection on the basis of Markov Decision Processes with fixed transition probabilities on the one hand, and scoring customers separately in pre-defined segments on the other. This points to a gap in the usual methodology insofar as customer scoring implies the explicit use of customer-specific information (covariates), while transition probabilities of Markov chains are conceived of as averages, without reference to the peculiarities of the customer to be addressed. Trying to unite both approaches, we suggest a model for customer transitions which allows transition probabilities to depend on covariates. Our model can be seen as an effort to focus on one-to-one marketing methods, permitting customer-specific action selection with the overall goal of customer value optimization. We show how to maximize the objective function subject to budget constraints. Our approach is motivated by the needs of a major European insurer. A numerical example with a realistic structure illustrates the capabilities of our approach.